Papers with task accuracy
KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference (2026.eacl-industry)
Copied to clipboard
| Challenge: | Long-context Large Language Models (LLMs) face significant memory bottlenecks due to the linear growth of key-value (KV) cache with sequence length. |
| Approach: | They propose a framework that maps the trade-off frontier between total memory consumption and task accuracy across three complementary optimization techniques. |
| Outcome: | The proposed model-specific configurations achieve 68-78% total memory reduction with minimal (1-3%) accuracy degradation on long-context tasks. |
BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents (2026.findings-acl)
Copied to clipboard
Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou
| Challenge: | Existing work on confidence in LLMs is limited. |
| Approach: | They propose to use confidence scores to determine model answer quality and encourage model to try again until it reaches satisfactory confidence level. |
| Outcome: | The proposed methods significantly reduce token consumption while demonstrating competitive performance compared to baseline fixed budget methods. |
Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation (2024.acl-srw)
Copied to clipboard
| Challenge: | Customizing LLMs for a specific task involves separating high-quality responses from lower-quality ones. Obtaining a large volume of expert-annotated data is costly for most tasks. |
| Approach: | They propose a method that trains the model to prioritize the best responses from a pool of candidates created for a task using ranking metrics. |
| Outcome: | The proposed method is more robust, less sensitive to noise, and can be achieved with limited human annotations or through heuristic methods. |
Garden Path Recovery in Causal and Masked Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | a linguistics study of garden-path sentences shows that recovery dynamics are important for linguistic evaluation . causal models show larger within-model disambiguation effects than masked models overall . |
| Approach: | They propose to compare garden-path recovery in causal and masked language models . they use 100 English garden- path/control pairs spanning three constructions . |
| Outcome: | The proposed model shows that decoder-only models exhibit sharper disruption at the point of syntactic revision, while encoders appear comparatively buffered at the disambiguator due to right-context access. |
Revisiting OPRO: The Limitations of Small-Scale LLMs as Optimizers (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies aim to enhance the efficacy of Large Language Models (LLMs) through strategic prompting. |
| Approach: | They propose to revisit the optimization by prompting approach for small-scale LLMs . they suggest future prompting engineering to consider both model capabilities and computational costs . |
| Outcome: | The proposed approach shows limited effectiveness in small-scale LLMs, with limited inference capabilities constraining optimization ability. |
GLaPE: Gold Label-agnostic Prompt Evaluation for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored leveraging the LLM itself as an optimizer to identify optimal prompts that maximize task accuracy. |
| Approach: | They propose a gold label-agnostic prompt evaluation method to reduce dependence on gold labels. |
| Outcome: | The proposed method produces more effective prompts even without gold labels. |
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge (2026.acl-srw)
Copied to clipboard
| Challenge: | Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediatereasoning that debate produces. |
| Approach: | They propose a framework that pairs a two-agent debate architecture with anLLM-as-judge that scores each agent’s reasoning along instructionfollowing, justification quality, and evidence grounding, together with acritical-failure flag. |
| Outcome: | The proposed framework pairs a two-agent debate architecture with anLLM-as-judge that scores each agent’s reasoning along instructionfollowing, justification quality, and evidence grounding, together with acritical-failure flag. |
Do LLM hallucination detectors suffer from low-resource effect? (2026.eacl-long)
Copied to clipboard
| Challenge: | a long line of work suggests that LLMs face issues along both dimensions . |
| Approach: | They investigate hallucination detectors' failure modes and their effects on the task accuracy of four LLMs and three halluciner detectors. |
| Outcome: | The models show impressive performance in high-resource languages like English but the performance degrades significantly in low-resourced languages like Bengali. |
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to condition models on a concise rationale are less accurate than models that can use the entire context. |
| Approach: | They propose a method to optimize a bound on the Information Bottleneck objective to extract concise rationales from a binary mask and an end-task predictor that uses only the residual sentences. |
| Outcome: | The proposed model outperforms existing norm-minimization techniques in task performance and agreement with human rationales in the ERASER benchmark. |
HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for visual token pruning lack insight into the intrinsic property of the vision encoder . et al., 2017: 99.3% of task accuracy with only 1/3 of the tokens. |
| Approach: | They propose a model-agnostic token pruning method that trains without training . they propose 'HiPrune' method which prunes visual tokens according to their attention . |
| Outcome: | The proposed method achieves 99.3% of task accuracy with only 1/3 of the tokens . it reduces inference FLOPs by 58.7% and maintains 99.99% accuracy with 2/9 tokens. |
Is This LLM Library Learning? Evaluation Must Account For Compute and Behaviour (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in the coding, reasoning, and tool-use ability of LLMs have raised the possibility of library learning with LLM. |
| Approach: | They propose to use reusable and composable functions and tools to create reusable, composesable code and tools that can be reused by modifying relevant examples. |
| Outcome: | The proposed system fails to consistently outperform the baseline model and does not correct for the difference in computational cost. |
Representations and Architectures in Neural Sentiment Analysis for Morphologically Rich Languages: A Case Study from Modern Hebrew (C18-1)
Copied to clipboard
| Challenge: | Existing sentiment analyzers for MRLs that use tokens and morpheme-based representations have no empirically studied effects of representation choices on neural sentiment analysis. |
| Approach: | They develop a sentiment analysis benchmark for Hebrew based on 12K social media comments and provide two instances of data. |
| Outcome: | The proposed benchmarks show that representation choices have measurable effects on task perfromance and that they vary depending on architecture type. |
Mitigating Biases in Language Models via Bias Unlearning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent debiasing approaches target different demographic groups, harming fairness and discrimination. |
| Approach: | They propose a model debiasing framework which targets stereotypes by unlearning stereotype forgetting and anti-stereotype retention. |
| Outcome: | The proposed framework outperforms existing methods in mitigating bias while retaining language modeling capabilities. |
Decompose and Compare Consistency: Measuring VLMs’ Answer Reliability via Task-Decomposition Consistency Comparison (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for estimating uncertainty using answer likelihoods or prompt-based confidence generation often suffer from overconfidence and confirmation biases. |
| Approach: | They propose to use Decompose and Compare Consistency (DeCC) to measure the reliability of a VLM's direct answer and indirect answers by decomposing the question into sub-questions and reasoning over the sub-answers. |
| Outcome: | Experiments on six vision-language tasks with three VLMs show that DeCC achieves better correlation with task accuracy compared to existing methods. |
Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting has been shown to enhance the multi-step reasoning capabilities of Large Language Models (LLMs). |
| Approach: | They propose to use CoT prompting to analyze a symbolic reasoning task where letters are shifted forward some number of steps in the alphabet. |
| Outcome: | The proposed model performs well on a symbolic reasoning task, with three LLMs performing the task using CoT prompts. |
Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization (2026.findings-acl)
Copied to clipboard
Xingjian Diao, Zheyuan Liu, Chunhui Zhang, Weiyi Wu, Keyi Kong, Lin Shi, Kaize Ding, Soroush Vosoughi, Jiang Gui
| Challenge: | Prior work has attempted to mitigate this issue by using adaptive reasoning strategies, but these methods overlook a fundamental bottleneck: visual perception failures. |
| Approach: | They propose a meta-reasoning controller that dynamically routes computation among three decision paths at each generation step. |
| Outcome: | The proposed method outperforms slow-thinking methods while producing shorter responses. |
MM-R3: On (In-)Consistency of Vision-Language Models (VLMs) (2025.findings-acl)
Copied to clipboard
| Challenge: | a flurry of research has been conducted on the performance of state-of-the-art (SoTA) Vision Language Models (VLMs) on a variety of tasks. |
| Approach: | They propose a benchmarking tool to analyze performance of SoTA Vision Language Models (VLMs) on three tasks: Question Rephrasing, Image Restyling, and Context Reasoning. |
| Outcome: | The proposed model achieves absolute improvements of 5.7% and 12.5% on widely used VLMs such as BLIP-2 and LLaVa 1.5M in terms of consistency over their existing counterparts. |
GeoLAN: Geometric Learning of Latent Explanatory Directions in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models lack transparency and are often unable to explain causal relationships . |
| Approach: | They propose a training framework that treats token representations as geometric trajectories and applies stickiness conditions to the Kakeya Conjecture. |
| Outcome: | The proposed training framework maintains task accuracy while improving geometric metrics and reducing fairness biases. |
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models (2022.emnlp-main)
Copied to clipboard
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
| Challenge: | Pre-trained Transformer models provide robust language representations which can be specialized on various tasks. |
| Approach: | They propose an efficient pruning method based on approximate second-order information that allows pruning weight blocks to be used for pruning. |
| Outcome: | The proposed method is the first to be applied at the BERT scale and significantly pushes the boundaries of the current sparse models with respect to all metrics: model size, inference speed and task accuracy. |
RCScore: Quantifying Response Consistency in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current evaluations of large language models rely on a single instruction template, overlooking models’ sensitivity to instruction style. |
| Approach: | They propose a multi-dimensional framework quantifying how instruction formulation affects model responses by transforming benchmark problems into multiple instruction styles. |
| Outcome: | The proposed framework reveals that instruction style can shift accuracy by 16.7% points. |
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained sequence to sequence models are effective in making and generating NL explanations, but they have many shortcomings. |
| Approach: | They propose a model that uses sentence markers to eliminate explanation fabrication . they use fusion-in-decoder architecture to handle long input contexts . |
| Outcome: | The proposed model significantly improves on the ERASER explainability benchmark. |
An LLM Feature-based Framework for Dialogue Constructiveness Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on dialogue constructiveness assessment focus on analysing conversational factors that influence individuals to take specific actions, win debates, change their perspectives or broaden their open-mindedness. |
| Approach: | They propose an LLM feature-based framework for dialogue constructiveness assessment that combines the strengths of feature- and neural approaches while mitigating their downsides. |
| Outcome: | The proposed framework outperforms standard feature-based models and neural models on three dialogue constructiveness datasets. |
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies demonstrate great performance gain for self-rationalization by few-shot prompting LMs with rationale-augmented exemplars. |
| Approach: | They propose to leverage explanations for small LMs to improve few-shot self-rationalization by reducing the problem of plausibility judgement to natural language inference. |
| Outcome: | The proposed approach achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric. |
Logical DA: Enhancing Data Augmentation for Logical Reasoning via a Multi-Agent System (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing data augmentation paradigms isolate data synthesis from label validation, thereby reducing their utility for complex reasoning tasks. |
| Approach: | They propose a framework for enhancing reasoning-focused data augmentation in few-shot learning scenarios that integrates four agents through two synergistic phases: diverse data generation and label verification. |
| Outcome: | The proposed framework achieves the highest average improvement in task accuracy in both fine-tuning and in-context learning paradigms. |
Adaptive Compression of Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks. |
| Approach: | They propose a method that uses a code-book approach to represent words as discrete codes such as (8, 5, 2, 4). |
| Outcome: | The proposed method makes the highly compressed word embeddings without hurting the task accuracy. |
Tandem Training for Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | As language models improve, their actions and reasoning will become difficult or impossible for weaker agents and humans to follow, undermining interpretability and oversight. |
| Approach: | They propose a tandem training paradigm that allows models to adapt their language to weaker partners by intermittently and randomly sampling a frozen weak model instead of the strong model being trained. |
| Outcome: | The proposed model is able to produce solutions that are intelligible to weaker agents and humans while keeping task accuracy high. |
ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning (2025.emnlp-main)
Copied to clipboard
Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Guanbo Wang, Fandong Meng, Jie Zhou, Ju Ren, Yaoxue Zhang
| Challenge: | Existing methods for fine-tuning-based compression suffer from verbose outputs, increasing computational overhead. |
| Approach: | They propose a framework to generate concise reasoning chains using Confidence Injection and Early Stopping. |
| Outcome: | The proposed framework reduces the length of the model by up to 50% while maintaining high task accuracy. |
Probing for Arithmetic Errors in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using a controlled setting of 3-digit addition, we show that simple probes can accurately decode both the model’s output and the correct answer from hidden states. |
| Approach: | They extend their analysis to structured chain-of-thought traces on addition-only GSM8K problems and find that probes trained on simple arithmetic generalize well to this more complex setting, revealing consistent internal representations. |
| Outcome: | The proposed probes can predict model correctness with over 90% accuracy on addition-only GSM8K problems and guide selective re-prompting of erroneous reasoning steps with minimal disruption to correct outputs. |
CAVE: Controllable Authorship Verification Explanations (2025.naacl-long)
Copied to clipboard
| Challenge: | Authorship Verification (AV) is used for tasks such as plagiarism detection, forensic analysis, analysis of the spread of misinformation. |
| Approach: | They propose to train an offline authorship verification model that is accessible and easy to use. |
| Outcome: | The proposed model generates high quality explanations and competitive task accuracy on three difficult AV datasets. |
IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method (2025.emnlp-main)
Copied to clipboard
Xinyu Liu, Bei Li, Jiahao Liu, Junhao Ruan, Kechen Jiao, Hongyin Tang, Jingang Wang, Tong Xiao, JingBo Zhu
| Challenge: | High-order numerical methods enhance performance in tasks like NLP but introduce a performance-efficiency trade-off due to increased computational overhead. |
| Approach: | They propose an iterative implicit Euler Transformer which simplifies high-order numerical methods by iterating implicit Eule. |
| Outcome: | The proposed method improves accuracy and reduces inference overhead by 55% while retaining 99.4% of the original task accuracy. |
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents (2026.acl-long)
Copied to clipboard
| Challenge: | a fundamental pillar of trustworthiness is calibration, which refers to an agent’s ability to express confidence that reliably reflects its actual performance. |
| Approach: | They propose a reinforcement learning framework that jointly optimizes task accuracy and calibration, supported by a holistic benchmark of reward designs. |
| Outcome: | The proposed framework improves calibration across tool types and shows that trained agents achieve superior calibration and exhibit robust generalization from local training environments to noisy web settings and to distinct domains such as mathematical reasoning. |
Waking Up Blind: Cold-Start Optimization of Supervision-Free Agentic Trajectories for Grounded Visual Perception (2026.findings-acl)
Copied to clipboard
| Challenge: | Small Vision-Language Models (SVLMs) suffer from visual brittleness and poor tool orchestration. |
| Approach: | They propose a supervision-free framework that bootstraps agentic capabilities via Coldstart Reinforcement Learning for SVLMs. |
| Outcome: | The proposed framework improves task accuracy and tool efficiency by 5% and 9%. |
Entropy-guided Vocabulary Augmentation of Multilingual Language Models for Low-resource Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models (MLLMs) support low-resource languages (LRLs) but LRL words are under-represented in wordpiece/subword vocabularies, leading to low task accuracy . |
| Approach: | They propose an entropy-based vocabulary augmented language model to detect LRL words with undesirable wordpiece segmentations. |
| Outcome: | The proposed model improves performance and limits on wordpiece augmentation strategies for multiple diverse LRLs. |
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge (2025.findings-emnlp)
Copied to clipboard
Yan-Lun Chen, Yi-Ru Wei, Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Ying-Dar Lin, Yu-Sung Wu, Wei-Bin Lee
| Challenge: | Large language models (LLMs) demonstrate strong task-specific capabilities through fine-tuning, but merging multiple fine- tuned models often leads to degraded performance due to overlapping instruction-following components. |
| Approach: | They propose a layer-wise approach that assigns layer-specific weights to task vectors based on their alignment with instruction-following or task-specific components. |
| Outcome: | The proposed approach outperforms existing methods in learning and forgetting tasks while preserving overall model utility. |
RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation (D19-1)
Copied to clipboard
| Challenge: | Existing work on map-based NL navigation relies on small artificial worlds with a fixed set of entities known in advance. |
| Approach: | They propose a task to interpret navigation instructions in natural language (NL) they use a dataset aligned with real, dense, urban maps to study neural architectures . |
| Outcome: | The proposed task is based on a dataset of 2515 navigation instructions aligned with real routes over three regions of Manhattan. |
Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate long-form and coherent text, yet they often hallucinate facts, which undermines their reliability. |
| Approach: | They propose a Learnable Intervention method for Truthfulness Optimization that automatically identifies the optimal intervention intensity tailored to each query context. |
| Outcome: | Experiments on multiple LLMs and question-answering datasets show that LITO improves truthfulness while preserving task accuracy. |
Uncovering and Categorizing Social Biases in Text-to-SQL (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Text-to-SQL models are trained on clean, neutral datasets, such as Spider and WikiSQl, but these models contain social bias at different rates. |
| Approach: | They propose to use data to map natural language utterances to SQL queries. |
| Outcome: | The proposed model can contain social bias at different rates in the downstream Text-to-SQL task. |
Don’t Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text (2023.acl-long)
Copied to clipboard
| Challenge: | ATINTER model can be used to rewrite adversarial inputs to make them non-adversarial . if undefended, model should maintain good task performance and effectively mitigate adversarials . |
| Approach: | They propose a model that intercepts adversarial inputs and learns to rewrite them . they show that it provides better adversarial robustness than existing defense approaches . |
| Outcome: | The proposed model improves adversarial robustness without compromising task accuracy on a sentiment classification dataset. |
Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing questionnaire-based evaluation methods exhibit limited stability and offer little explainability, as their results are sensitive to minor variations in prompt phrasing or role-play configurations. |
| Approach: | They propose an internal-activation-based approach for stable and explainable personality trait evaluation in Large Language Models by interpolating a persona vector associated with a target personality trait from the model's internal activations. |
| Outcome: | The proposed approach yields significantly more stable personality trait evaluations than existing methods, even under questionnaire and role-play variants. |
Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks prioritize structural architectures and role assignments but neglect granular mechanics of agent collaboration. |
| Approach: | They propose to use centralized governance, instructor-led participation, ordered interaction patterns to optimize task accuracy and computational efficiency. |
| Outcome: | The proposed model improves task accuracy and computational efficiency under two context-dependent scenarios. |
SaGE: Evaluating Moral Consistency in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on Large Language Models (LLMs) have focused on accuracy but lack universally agreed-upon answers for moral scenarios. |
| Approach: | They propose a measure called Semantic Graph Entropy to measure a model's moral consistency grounded in "Rules of Thumb" they construct a moral Consistency Corpus (MCC) with 50K moral questions and the RoTs they followed to investigate LLM consistency on two popular datasets. |
| Outcome: | The proposed measure measures moral consistency on two popular datasets . |
Refining Attention for Explainable and Noise-Robust Fact-Checking with Transformers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Conventional transformer-based models falter due to noise sensitivity and lack explainability . ATTUN is a transformer architecture designed to enhance model transparency and resilience to noise. |
| Approach: | They propose a transformer architecture that enhances model transparency and resilience to noise . ATTUN is a module that directly modifies attention weights . they validated their approach using fact-checking datasets based on their results . |
| Outcome: | The proposed model improves predictions and identify relevant sections of input data. |
Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | despite growing interest in explainable NLP, it remains unclear how explanation strategies shape user behavior in tasks like authorship identification. |
| Approach: | They propose two explanation types to support their analysis of user behavior . they use example-based style rewrites and feature-based rationales to generate explanations . |
| Outcome: | The proposed explanations support appropriate reliance, whereas explanations increase AI overreliance, the study finds . |
PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction (2026.findings-acl)
Copied to clipboard
| Challenge: | Current benchmarks evaluate task accuracy but overlook how agents interact . Preference-aware agents show 7.6% average UX improvement and 18.5% gain in preference alignment. |
| Approach: | They propose a configurable environment that evaluates both what agents accomplish and how they interact. |
| Outcome: | The proposed model improves performance and improves user experience by 7.6% and 18.5% respectively. |
From Individual Excellence to Collective Sustainability: Seeking Strategic Equilibrium in Proactive Multi-Agent Teams (2026.findings-acl)
Copied to clipboard
| Challenge: | a team of proactive agents suffer from a greedy optimization for immediate task accuracy . a new approach to improve team collaboration is based on the opportunity cost . |
| Approach: | They propose a game-theoretic proactive multi-agent reinforcement learning framework to solve this imbalance . they use a Positive-Unlabeled scorer to anchor intervention quality under sparse supervision . |
| Outcome: | The proposed framework maintains high performance while preventing experts from over-developing. |
From Tasks to Teams: A Risk-First Evaluation Framework for Multi-Agent LLM Systems in Finance (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on task specific metrics such as accuracy, F1 score, or ROUGE. |
| Approach: | They propose a multi-agent, safety-aware evaluation agent that audits large language models without fine-tuning. |
| Outcome: | M-SAEA identifies unsafe trajectories with minimal false positives and reveals latent risks that are not addressed by standard metrics. |
Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging (2026.findings-acl)
Copied to clipboard
| Challenge: | We show that the "alignment tax" of post-training is framed as a drop in task accuracy. |
| Approach: | They propose a more holistic view of the alignment tax by framing it as a drop in accuracy and a degradation of model calibration. |
| Outcome: | The proposed method improves accuracy beyond both parents while recovering calibration lost during alignment. |